Does MSFT DiskANN repo not support ARM?

Yes, they have mkl-devel as a dependency, which is not meant to be installed in aarch64.

Where is this defined? Is it using the CPU DiskANN repo to build the ssd verson of the index?

Yes, that is right. It is just to benchmark the CPU build

@achirkin if we do not have this line, the entire dataset will be read into the base_set variable needlessly for DiskANN ssd based indexes. For these indexes, only the path to the dataset needs to be known and the DiskANN index building APIs will read the data from the path. In fact, it is important we do it this way because those SSD indexes are designed to let the data remain on disk if there are extremely large datasets.

You can just set the dataset_memory_type to kHostMmap by default for diskann index. It will lazily map the file content, so it won't read anything until you actually use the pointer.
It will still open the file and read the first two words anyway - independently of this change. That is because the benchmark reads and reports the dataset dimensionality.

You can just set the dataset_memory_type to kHostMmap by default for diskann index. It will lazily map the file content, so it won't read anything until you actually use the pointer.
It will still open the file and read the first two words anyway - independently of this change. That is because the benchmark reads and reports the dataset dimensionality.

I think you shouldn't need path_to_index parameters, because the index member file is accessible from within the index in your algorithm-specific code.
If you need the dataset file path during index build, you can either ingest it as a build param in the conf.hpp, similar to how we inject some search parameters in parse_index(...), or create one more member in the configuration::index - all in conf.hpp; I think it's ok to have it available to all algorithm at that point.

Unfortunately, the cuvs::bench::configuration::index object's attributes are not directly passed down to the specific algorithm's wrapper class. So we would need the path_to_index. The algorithm specific structs use only the parameters that are explicitly passed down from the configuration object.

My question is: CAN we pass the cuvs::bench::configuration::Index down into the wrappers? Or rather, any reason we shouldn't do it?

Same problem, the algorithm-specific code should go to the algorithm headers.

This is just for checking if the index file exists. The diskann ssd index is saved with the suffix _disk.index along with the filename.

This is algorithm-specific code in common benchmarking code, though. This should be moved into the diskann portions of the code. You have to understand that we create generalized abstractions for this very purpose- anything in common/benchmark.hpp should be applied to all index algorithms and any specific things should be propagated down to the respective index types. Polluting the common code like this moves up the abstraction tree, rather than down, and it results in hard to maintain spaghetti code.

This is another reason to propagate this down to the individual wrappers. You could at least specify a method on the wrapper class to allow any algorithms to override and adjust the filename as needed. That would be more robust than one-offs like this.